Assessing Agreement on Segmentations by Means of Staccato, the Segmentation Agreement Calculator according to Thomann

Staccato, the Segmentation Agreement Calculator According to Thomann, is a software tool for assessing the degree of agreement of multiple segmentations of some time-related data (e.g., gesture phases or sign language constituents). The software implements an assessment procedure developed by Bruno Thomann and will be made publicly available. The article discusses the rationale of the agreement assessment procedure and points at future extensions of Staccato. 1 Segmentation and Annotation in Gesture Studies No matter what your favourite definition of co-verbal “hand and arm gesture” is, at the basic kinematic level gestures are manifested by spatio-temporal bodily movements. However, not every body movement constitutes gestural behaviour. Accordingly, a prerequisite of any empirical, data-driven account to gestures is to locate those segments in the continuous stream of hand and arm movements that make up the gestures. Furthermore, gestures are structured in themselves: a gesture basically has a tripartite movement profile, consisting of a preparation, a stroke, and a retraction phase [5]. Therefore, gestural hand-and-arm movements not only are to be identified, but are also to be subdivided into phases. In addition, temporary cessations of motion might occur either before or after the stroke phase – the so-called preand post-stroke holds [7]. The demarcation of the temporal parts of a motion that constitutes a gesture and its phases is referred to as the segmentation problem throughout this article. The segmentation problem is not confined to gesture studies, of course; it also prevails in sign language research, and, generically, in accounts to temporally organised behaviour in general (think, e.g., of phoneticians that identify phonemes in a sound stream, choreographers that decompose dance figures, speech pathologists that demarcate deglutition phases, or traffic authority employees studying rush hour traffic). We focus on empirical work on gesture, however, where segmentation is typically carried out on video recordings, in which the significant movements are to be demarcated with respect to the video’s time line. The segmentation problem has to be kept apart from the annotation problem. In an annotation session, the annotator (or coder, or rater) has to classify items according to a set of response categories. A common example is that of nurses that have to assess the psychological health state of patients, say, in terms of “happy”, “stoic”, and “sad”. In case of gesture studies, the “patients” are gestures that are classified according to categories defined in a classification scheme. For instance, in case of the Speech-and-Gesture Alignment corpus (SaGA [8]) one set of response categories has been derived from the so-called representation techniques [9], that is, from the displaying functions gestures employ in order to depict something, like, for instance, drawing or locating. item classification 1 A 2 B 3 A 4 B 5 C .. .. n cn data sream                  segmentation                   